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ABSTRACT 


By  using  graph-theoretic  techniques  to  compare  the 
information  processing  behaviors  of  three  groups  of  mid¬ 
level  working  professionals  as  each  undertakes  a  series  of 
four  complex,  interdependent,  computer-mediated  decision¬ 
making  exercises,  this  thesis  explores  1)  the  relationship 
between  network  centrality  and  individual  performance  and  2) 
the  relationship  between  network  density  and  group 
performance.  The  results  of  this  exploration,  though  mostly 
inconclusive,  call  into  question  both  intuition  and  social 
network  analysis  literature.  It  is  predicted  that  centrality 
in  a  network  correlates  positively  with  high  performance 
among  individuals,  but  statistical  analysis  of  data 
collected  during  controlled  experimentation  reveal  an  almost 
negligible  relationship.  It  is  also  hypothesized  that  high 
density  groups  outperform  low  density  groups,  but  density 
and  performance  are  found  to  correlate  in  exactly  the 
opposite  direction:  as  density  increases,  group  performance 
decreases . 

As  an  explanation,  this  thesis  proposes  that  as  network 
density  increases  actors  require  more  time  to  process  and 
respond  to  incoming  information.  In  as  much  as  central 
actors  possess  a  greater  number  of  edges  (i.e., 
communication  linkages  to  others) ,  this  thesis  also  argues 
that  centrality  in  a  network  has  costs,  as  well  as  benefits. 
Further  experimentation  is  needed  to  test  the  validity  of 
these  conjectures  and  bring  better  understanding  to 
Organization  Theory,  Social  Network  Analysis,  and 
Information  Processing  networks. 
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I .  INTRODUCTION 


Although  many  studies  have  been  undertaken  which  study 
the  performance  of  organizations,  the  virtues  of  a  computer 
mediated  experiment— increased  control  over  data  collection, 
increased  volume  of  data  collected,  and  increased 
computational  power— enable  the  researcher  in  certain 
instances  to  make  more  confident  assertions  about 
organizational  behavior  than  he  otherwise  is  enabled  using 
different  techniques.  In  order  to  explore  the  dynamic 
transactions  that  occur  within  an  organization,  beyond  its 
macro  level  design  features,  over  the  years  social  network 
analysis  scholars  have  developed  powerful  graph-theoretic 
techniques  which  allow  researchers,  in  one  sense,  to  look 
under  the  hood  of  an  organization  and  observe  its  inner 
workings.  This  thesis  uses  these  techniques  to  explore  1) 
the  relationship  between  network  centrality  and  individual 
performance  and  2)  the  relationship  between  network  density 
and  group  performance. 

A.  PURPOSE  AND  MOTIVATION 

In  as  much  as  one  encounters  organizations  unsuited  to 
their  information  processing  requirements,  studies  of  how 
these  organizations  adapt  to  their  adverse  environment  has 
the  potential  to  highlight  what,  if  any,  differences  in 
communications  processing  patterns  can  be  deemed  beneficial. 
A  great  many  organizations  today,  transitioning  from  the 
Industrial  Age  into  the  Information  Age,  possess  structures 
that  are  unsuited  to  their  environment.  As  these 
organizations  labor  to  change  the  "anti-sharing,  anti- 
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collaborative"  behaviors  typical  of  hierarchies  and 
bureaucracies,  society  continues  to  demand  that  they 
function  in  their  weakened  state  whether  in  industry, 
government,  or  defense.1  It  has  been  argued  that  the 

mechanistic  form  of  organization  that  evolved  from  and  so 
pervaded  the  Industrial  Age  will  be  unable  to  handle  the 
challenges  of  the  future.2 

In  some  cases  these  organizations  will  meet  with 
success,  as  certain  subunits  adapt  to  changing  information 
processing  requirements.  The  failures  of  other 
organizations,  however,  will  result  in  the  need  for  further 
reevaluation  and  subsequent  restructuring.  It  is  therefore 
important  to  obtain  some  understanding  of  the 

characteristics  of  a  successful,  yet,  disadvantaged 
organization  as  it  struggles  with  a  demanding  environment  to 
which  it  is  unsuited.  This  thesis  is  an  attempt  to 
understand  these  attributes,  though  from  a  laboratory  based 
computer  mediated  exercise,  rather  than  first  hand  field 
studies  and  observations.  Whereas  organizations  sometimes 
operate  in  information  processing  environments  to  which  they 
are  not  suited,  this  thesis  seeks  to  uncover  through  the  use 
of  graph-theoretic  techniques,  what  social  network  analysis 
measures  correlate  with  high  performance. 

B .  METHOD 

From  the  10th  of  January  until  the  15th  of  February, 
2007,  a  series  of  experiments  took  place  in  the  Information 

1  David  S.  Alberts  and  Richard  E.  Hayes,  Power  to  the  Edge  (Command 
and  Control  Research  Project,  2003),  73. 

2  Simon  R.  Atkinson  and  James  Moffat,  The  Agile  Organization  (Command 
and  Control  Research  Project,  2005),  158-159. 
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Sciences  Department  of  the  Naval  Postgraduate  School  that 
examined  the  performance  of  two  distinct  types  of 
organizations:  the  Hierarchy  and  the  Edge.  The  experiment 
details  are  summarized  in  Leweling  and  Nissen  (2007) . 
Although  the  hypotheses  tested  in  this  thesis  were 
formulated  after  the  design  and  execution  of  the  experiment, 
the  data  collected  were  so  rich  as  to  enable  exploration 
into  topics  not  originally  conceived  as  part  of  the 
experiment . 

C.  ANALYSIS 

For  this  thesis,  the  author  coded  data  from  the  log 
files  generated  during  the  ELICIT  experiment  into  time 
demarcated  adjacency  matrices  which  are  then  evaluated  with 
the  Organizational  Risk  Analyzer  (ORA) ,  a  social  network 
analysis  software  application  developed  at  Carnegie  Mellon 
University.  By  means  of  the  software  these  data  are 
converted  into  reliable  measures  of  centrality  and  density, 
the  independent  variables  analyzed  in  this  thesis. 
Performance,  in  terms  of  time  and  accuracy,  are  the 
dependent  variables  of  this  thesis.  Finally,  the  independent 
and  dependent  variables  are  checked  for  correlations. 

D.  ORGANIZATION  OF  THESIS 

The  following  chapter  presents  a  succinct  survey  of 
relevant  Organizational  Theory  literature  that  will  begin 
with  Mintzberg  (1983)  and  terminate  with  Nissen,  drawing 
lightly  upon  Burns  and  Stalker  (1961)  and  Lawrence  and 
Lorsch  (1967)  .  Chapter  II  concludes  with  the  statement  of 
hypotheses.  Chapter  III  begins  with  an  introduction  to  the 
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ELICIT  experiment  that  provides  a  description  of  its 
environment,  purpose,  and  organization;  it  is  an  explanation 
of  how  ELICIT  attempts  to  operationalize  and  compare  the 
Hierarchy  and  Edge  Organizations.  This  chapter  also 
delineates  the  measures  used  to  determine  an  agent' s  success 
within  a  complex  and  interdependent  information  processing 
environment.  Chapter  IV  details  the  statistical  results  of 
the  research  with  respect  to  the  hypotheses  under 
investigation.  Chapter  V  summarizes  these  results  and  offers 
an  attempt  to  bridge  the  divide  between  reality  and  the 
results  of  this  computer  mediated  decision  making  exercise, 
explaining  the  real  world  applicability  of  its  conclusions. 
The  thesis  ends  with  a  brief  summary  and  suggestions  for 
future  research  opportunities. 
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II.  LITERATURE  REVIEW 


In  order  to  ground  the  research  and  analysis,  this 
chapter  begins  with  a  brief  review  of  organizational  theory, 
and  in  particular,  information  processing  as  a  framework  for 
viewing  organizations.  We  also  provide  fixed  definitions  for 
Edge  and  Hierarchy  forms  to  permit  disciplined  exploration 
of  these  forms. 

A.  ORGANIZATIONAL  THEORY 

Any  study  of  organization  theory  requires  that  the  one 
be  familiar  with  the  major  points  of  Henry  Mintzberg' s 
seminal  work.  Structures  in  Fives:  Designing  Effective 
Organizations .  This  book  represents  a  basis  for  the  present 
investigation . 

1 .  The  Five  Coordinating  Mechanisms 

Mintzberg  (1983)  posits  that  five  ways  exist  in  which 
organizations  can  coordinate  their  work:  1)  mutual 
adjustment,  2)  direct  supervision,  3)  standardization  of 
work  processes,  4)  standardization  of  outputs,  and  5) 
standardization  of  skills.  Mutual  adjustment  accomplishes 
its  end  by  means  of  informal  communication  that  takes  place 
between  "doers."  Mintzberg  (1983)  states  that  although 
mutual  adjustment  is  used  in  the  simplest  of  organizations, 
"paradoxically,  it  is  also  used  in  the  most  complicated."3 
The  challenges  facing  an  organization  in  the  most  complex  of 
environments  are  often  unknown  or  vaguely  understood, 

3  Henry  Mintzberg,  Structures  in  Fives:  Designing  Effective 
Organizations  (Upper  Saddle  River,  New  Jersey:  Prentice  Hall,  1983),  4. 
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requiring  that  people  within  the  organization  "adapt  along 
their  uncharted  route."4  A  slightly  larger  organization 
requires  a  different  type  of  coordinating  mechanism:  direct 
supervision.  In  this  case,  one  individual  is  placed  in 
charge  of  several  others,  and  spends  his  time  giving 
instructions  to  his  subordinates  and  monitoring  their 
actions . 

Work  can  further  be  coordinated  by  standardization. 
Work  processes  are  standardized  when  an  organization 
provides  its  laborers  with  a  clear  set  of  instructions  for 
them  to  follow.  Fast  food  restaurants  are  an  example,  where 
employees  fill  food  orders  in  an  assembly  line  fashion  and 
are  successful  to  the  extent  that  they  do  not  deviate  from 
the  thorough  set  of  instructions  provided  by  management. 
When  an  organization  produces  many  different  goods  and 
services  consolidation  is  achieved  by  standardization  of 
outputs.  For  multinational  corporations  the  standardization 
is  focused  on  growth  in  earnings. 

Finally,  work  can  be  coordinated  by  means  of 
standardization  of  skills  and  knowledge.  In  the  case  of 
universities,  for  example,  professors  teaching  in  a 
classroom  possess  the  training  and  knowledge  necessary  to 
educate.  Mintzberg  (1983)  notes  that,  "As  an  organization'’ s 
work  becomes  more  complicated,  the  favored  means  of 
coordination  seems  to  shift  from  mutual  adjustment  to  direct 
supervision  to  standardization...  finally  reverting  back  to 
mutual  adjustment."5 6 


4  Mintzberg,  Structures  in  Fives,  4. 

Mintzberg,  Structures  in  Fives,  4-7. 
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2. 


The  Five  Elements  of  Organizations 


In  Mintzberg' s  model,  the  five  elements  of  an 
organization  are:  1)  operating  core,  2)  strategic  apex,  3) 
middle  line,  4)  technostructure,  and  5)  support  staff. 

Relating  to  the  production  of  products  and  services, 
the  operating  core  is  concerned  most  directly  with  the  basic 
work  of  the  organization.  The  operators  have  four  primary 
functions:  to  1)  secure  inputs,  2)  transform  inputs  to 
outputs,  3)  distribute  outputs,  and  4)  provide  direct 
support  to  the  input,  output,  and  transformation  functions.6 

Although  the  operating  core  is  the  heart  of  every 
organization,  in  order  to  function  effectively  organizations 
require  an  administrative  apparatus  as  well.  The  head  of  the 
administrative  component  is  the  strategic  apex,  which 
ensures  that  the  organization  carry  out  its  mission  in  an 
effective  way,  and  fulfill  the  desires  of  those  who  control 
or  have  power  in  the  organization,  such  as  its  stockholders, 
employees,  or  donors.  Concurrently,  the  strategic  apex  has 
three  sets  of  duties:  1)  to  the  extent  that  the  organization 
relies  on  direct  supervision,  they  allocate  resources,  issue 
orders,  authorize  major  decisions,  resolve  conflicts 
etcetera,  2)  manage  the  organization's  relation  with  the 
outside  environment,  and  3)  develop  strategy,  the  mediating 
force  between  the  organization  and  its  environment.7 
Connecting  the  strategic  apex  with  the  operating  core  is  the 
middle  line.  When  an  organization  relies  heavily  on  direct 


6  Mintzberg,  Structures  in  Fives,  12. 

7  Mintzberg,  Structures  in  Fives,  13. 
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supervision,  the  middle  line  tends  to  grow  quite  large, 
creating  an  organizational  hierarchy. 

On  the  other  hand,  the  technostructure  formulates 
certain  methods  of  standardization  for  the  organization.  It 
is  composed  of  analysts  who  are  removed  from  the  work  flow, 
but  may  design  it,  or  train  people  for  it.  Finally,  the 
support  staff  exists  to  provide  services  to  the  members  of 
an  organization  that  are  not  involved  with  operating  work 
flow,  for  instance:  payroll  and  janitorial  services.8 

3 .  The  Five  Typologies  of  Organizations 

In  Mintzberg' s  model,  organizations  possess  one  of  five 
distinct  typologies:  1)  Simple  Structure,  2)  Machine 
Bureaucracy,  3)  Professional  Bureaucracy,  4)  Divisional 
Form,  and  5)  Adhocracy. 

As  the  name  suggests,  the  Simple  Structure  is  the  least 
elaborate  of  the  five  forms  of  organization.  Generally,  it 
has  little  formalized  behavior  and  few  support  staff,  lacks 
a  technostructure  and  a  strict  division  of  labor,  and  has  a 
small  managerial  hierarchy.  It  is  the  most  organic  of 
forms . 9 

Most  clearly  the  object  of  this  thesis,  the  Machine 
Bureaucracy  is  characterized  as  highly  formalized,  highly 
specialized,  and  hierarchical.  Because  this  form  of 
organization  predominately  relies  on  formalization  of 
processes  for  coordination,  the  technostructure  becomes 
essential  to  the  organization's  successful  operation.10 

8  Mintzberg,  Structures  in  Fives,  15-16. 

9  Mintzberg,  Structures  in  Fives,  157-158. 

Mintzberg,  Structures  in  Fives,  164-165. 
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During  the  Industrial  Era,  the  Machine  Bureaucracy  emerged 
as  the  organizational  form  most  well  suited  to  the 
environment  of  that  time.11  Its  structure  repeatedly 
demonstrated  the  ability  to  mass  produce  goods,  run 
governments,  and  fight  wars. 

The  third  typology  Mintzberg  (1983)  considers  is  the 
Professional  Bureaucracy  which  relies  heavily  upon  the 
training  and  indoctrination  necessary  to  coordinate  by  means 
of  standardization  of  skills.  Doctors,  teachers,  and 
accountants  in  effect  learn  what  to  expect  from  their 
counterparts  within  the  Professional  Bureaucracy.12 

The  key  element  of  the  Divisional  Form  of  organization 
is  the  middle  line.  Each  division  operates  with  near 
autonomy  within  its  highly  circumscribed  area,  using 
standardization  of  outputs  to  achieve  coordination. 

The  last  typology  Mintzberg  (1983)  addresses  is  the 
Adhocracy,  which,  similar  to  the  Simple  Structure,  achieves 
coordination  by  mutual  adjustment.  He  considers  this  form  to 
be  highly  organic  with  little  formalization,  agile  in 
complex,  dynamic  environments,  and  the  "least  reverent  for 
classical  principles  of  management,  especially  unity  of 
command .  "13 

4 .  The  Eight  Design  Factors 

Mintzberg  (1983)  also  includes  a  description  of  eight 
design  factors  necessary  to  the  workings  of  any 
organization.  They  are:  1)  job  specialization,  2)  behavioral 

11  Alberts  and  Hayes,  Power  to  the  Edge,  37-51. 

12  Mintzberg,  Structures  in  Fives,  190. 

Mintzberg,  Structures  in  Fives,  253-254. 
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formalization,  3)  training  and  indoctrination,  4)  unit 
grouping,  5)  unit  size,  6)  planning  and  control  systems,  7) 
liaison  services,  and  8)  decentralization. 

Job  specialization  possess  both  a  horizontal  component- 
how  many  different  types  of  tasks  a  worker  accomplishes— and 
a  vertical  component— how  much  control  the  work  has  over  the 
"how"  and  "why"  of  his  tasks.  Organizations  formalize 
behavior  in  three  ways:  by  position,  by  work  flow,  and  by 
rules.  Training  and  indoctrination  refer  the  process  by 
which  organizations  impart  job  related  skills  and 
organizational  norms. 

Unit  grouping  provides  four  important  effects  to  an 
organization:  it  establishes  a  system  of  command  supervision 
among  positions  and  units,  requires  positions  and  units  to 
share  common  resources,  creates  a  common  measure  of 
performance,  and  encourages  mutual  adjustment.  Unit  size 
affects  the  coordination  mechanism  most  suited  to  an 
organization . 

Liaison  services  and  planning  and  performance  controls 
"grease  the  wheels  of  mutual  adjustment."14  Lastly,  when  the 
power  to  make  decisions  is  widely  dispersed  throughout  an 
organization,  the  organization  is  said  to  be 
decentralized. 15 

B.  INFORMATION  PROCESSING  MODEL 

In  a  step  away  from  Mintzberg  (1983)  and  the  number 
five,  Tushman  and  Nadler  (1978)  choose  instead  to  view 
organizations  as  information  processing  systems.  The 

14  Mintzberg,  Structures  in  Fives,  73. 
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information  processing  model  is  based  on  assumptions  derived 
from  well  accepted  organization  theory  literature.  First, 
Tushman  and  Nadler  (1978)  assert  that  organizations  are  open 
social  systems  which  must  deal  with  work-related 
uncertainty.  Dependent  on  inputs  from  the  environment  in 
which  they  are  situated,  organizations  may  have  to  process 
uncertainties  that  arise  from  sources  beyond  their  control. 
Additionally,  uncertainties  emerge  from  within  an 
organization  as  a  result  of  a  multitude  of  human  factors.  In 
order  to  cope  with  both  external  and  internal  sources  of 
uncertainty,  organizations  must  develop  information 
processing  mechanisms  which  translate  multifarious  forms  of 
data  into  timely,  accurate,  concise,  and  relevant 
information . 

With  this  in  mind,  Tushman  and  Nadler  (1978)  affirm 
that  it  is  beneficial  to  view  organizations  as  information 
processing  systems.  Finally,  they  claim  that  organizations 
are  composed  of  subunits,  each  having  to  deal  with  a  varying 
degree  of  uncertainty  relative  to  the  its  position  within 
the  organization's  information  processing  structure  and  the 
external  environment.16  From  these  assumptions,  Tushman  and 
Nadler  (1978)  derive  a  series  of  five  propositions,  the  last 
four  of  which  are  salient  to  this  thesis: 

1 .  Proposition  1 

The  tasks  of  organizational  subunits  vary  in  their 
degree  of  uncertainty. 

16  Mintzberg,  Structures  in  Fives,  25-95. 

16  Tushman  and  Nadler,  "Information  Processing  as  an  Integrating 
Concept  in  Organizational,"  The  Academy  of  Management  Review  (July, 

1978),  615-616. 
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A  subunit' s  degree  of  uncertainty  and  therefore  of 
information  processing  reguirements  results  from  subunit 
task  characteristics,  subunit  task  environment,  and  inter¬ 
unit  task  interdependence.17 

2 .  Proposition  2 

As  work  related  uncertainty  increase,  so  does  the  need 
for  increased  amounts  of  information,  and  thus  the  need  for 
information  processing  capacity. 

When  subunits  face  little  uncertainty,  they  need  only 
pass  on  the  barest  of  information  to  their  superiors  and 
other  subunits  in  the  organization.  However,  when  subunits 
face  a  high  degree  of  uncertainty,  they  must  transmit  to  the 
organization  a  correspondingly  high  degree  of  information. 
This  additional  information  requires  that  the  organization 
as  a  whole  is  capable  of  processing  more  information.18 

3 .  Proposition  3 

Different  organizational  structures  have  different 
capacities  for  effective  information  processing. 

While  certain  stable  environments  may  be  conducive  to 
mechanistic  organizational  forms,  organizational  theory 
literature  generally  attributes  a  higher  degree  of 
information  processing  capacity  to  more  organic  forms  of 
organization.  The  literature  indicates  that  more  connected 
forms  of  organizations  are  better  able  to  handle 
uncertainty,  permitting  individuals  to  mutually  adjust  to 
solve  problems  and  error  correct.  Despite  being  better  able 

17  Tushman  and  Nadler,  The  Academy  of  Management  Review,  615. 

18  Tushman  and  Nadler,  The  Academy  of  Management  Review,  616-617. 
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to  handle  higher  levels  of  information  flow,  organic 
organization  structures  expend  more  time  and  energy  to 
achieve  their  solutions.  Thus,  the  choice  of  structure  must 
balance  need  with  cost.19 

4 .  Proposition  4 

Organizations  will  be  more  effective  when  there  is  a 
match  between  information  processing  requirements  facing  the 
organization  and  information  processing  capacity  of  the 
organization . 

In  the  case  of  an  organization  with  not  enough 
information  processing  capacity  for  its  environment,  Tushman 
and  Nadler  (1978)  expect  performance  to  be  degraded. 
Likewise,  if  an  organization  over  invests  in  information 
processing  capacity,  they  assert  that  performance  will  be 
less  than  optimal.  Only  when  structure  matches  capacity  will 
there  be  a  match  that  is  both  efficient  and  optimal.20  These 
assertions  are  diagramed  in  Table  1. 


19  Tushman  and  Nadler, 

20  Tushman  and  Nadler, 


The  Academy  of  Management  Review,  617-618. 
The  Academy  of  Management  Review,  619-620. 
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Information 

Processing 

Requirements 

Information  Processing  Capacity 

High 

Low 

Extensive 

Match 

Mismatch 

A 

B 

Minimal 

Mismatch 

Match 

C 

D 

Table  1.  Relationships  Between  Information  Processing  Capacity 
and  Information  Processing  requirements.  From  Tushman 

and  Nadler  21 


5 .  Proposition  5 

If  organizations  face  different  conditions  over  time , 
more  effective  units  will  adapt  their  structures  to  meet  the 
changed  information  processing  requirements. 

Existing  in  a  dynamic,  evolving  world,  organizations 
will  not  always  face  the  same  environment  in  which  they  were 
formed.  As  their  environment  changes,  so  too  does  the  need 
for  information  processing  capacity,  and  therefore 
organizations  must  adapt  their  information  processing 
structure  in  order  to  remain  fit. 


21  From  Tushman  and  Nadler,  The  Academy  of  Management  Review,  619. 
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C.  ORGANIZATION  PERFORMANCE  AND  ENVIRONMENTAL  CHANGE 

Although  macro  level  organizational  design  factors 
certainly  influence  group  performance,  Pearce  and  David 
(1983)  argue  that  they  do  so  indirectly— through  information 
flow— and  that  group  level  characteristics  serve  to  moderate 
performance.22  Unearthing  these  group  level  characteristics 
is  possible  through  use  of  social  network  analysis  to  study 
information  exchange  and  influence  relationships,  an 
analytical  technique  specifically  recommended  by  Pearce  and 
David  (1983)  .  Social  network  properties  have  been  compared 
to  measures  of  performance,  and  some  organization  theory 
scholars  have  suggested  that  that,  among  others,  the 
following  group  structural  properties  positively  impact 
performance:  1)  high  centrality,  2)  few  coalitions,  3)  few 
isolates,  4)  many  stars,  5)  many  liaisons,  6)  high 
connectedness,  and  7)  high  reciprocity.  Additionally,  these 
properties  are  believed  to  negatively  impact  performance:  1) 
many  coalitions,  2)  many  isolates,  3)  few  stars,  4)  few 
liaisons,  5)  low  connectedness,  and  6)  low  reciprocity.23 

It  is  not  clear  at  this  time  the  extent  to  which 
today's  organizations  will  confront  a  more  complex 
information  processing  environment  than  the  environment  in 
which  they  developed.  But  clearly,  the  world  is  not  static, 
and  the  increasing  complexity  noted  by  many  scholars 
associated  with  "globalization,  technology,  hyper¬ 
competition,  and,  knowledge-based  innovation"  will  force 

22  John  A.  Pearce  II  and  Fred  R.  David,  "A  Social  Network  Approach  to 
Organizational  Design-Performance, "  Academy  of  Management  Review  (July, 
1983),  436. 

23  Pearce  and  David,  Academy  of  Management  Review,  441. 
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ill-structured  organizations  into  a  more  dynamic,  unstable 
environment.24  As  the  macro  level  design  features  hinder 
certain  organizations  as  they  attempt  to  meet  the 
increasingly  extensive  information  processing  reguirements 
of  the  future,  one  is  still  likely  to  find  strong 
performance  among  groups  and  individuals  defined  by  the 
aforementioned  structural  properties. 

D.  USING  NETWORK  ANALYSIS  TO  EXPLORE  ORGANIZATIONS 

Organizational  theorists  choose  to  deconstruct  and 
analyze  organizations  in  many  ways,  including  in  terms  of 
structure  and  macro  level  design  features.  While  this 
approach  allows  theorists  to  develop  constructs  concerning 
complexity,  formalization,  centralization,  technology,  and 
size,  Fombrun  and  Tichy  (1979)  argue  that  such  a  perspective 
by  itself  is  insufficient  for  capturing  the  dynamic  realty 
of  organizations.25  With  this  in  mind,  social  network 
analysis  attempts  to  enrich  a  limited  and  static  viewpoint 
by  empirical  measures  of  the  behavior  of  individual  actors 
within  an  organization.  And  yet,  organizational  theorists 
are  unable  to  incorporate  social  network  analysis  without  a 
debate  concerning  the  nature  of  networks  as  they  relate  to 
organizations . 


24  Tara  A.  Leweling  and  Mark  E.  Nissen,  "Hypothesis  Testing  of  Edge 
Organizations:  Laboratory  Experimentation  using  the  ELICIT  Multiplayer 
Intelligence  Game"  12th  International  Command  and  Control  Research  and 
Technology  Symposium:  Adapting  C2  to  the  21st  Century  (Newport,  RI  June, 
2007),  2. 

25  Charles  Fombrun  and  Noel  Tichy,  "Network  Analysis  in 
Organizational  Settings,"  Human  Relations  (1979),  926. 


16 


Some  scholars,  such  as  Podolny  and  Page  (1991),  Bovasso 
(1992),  and  van  Alstyne  (1997)  view  networks  as  a 
fundamentally  new  type  of  organization  defined  as: 

Any  collection  of  actors  (N>2)  that  pursue 
repeated,  enduring  relations  with  one  another 
and,  at  the  same  time,  lack  a  legitimate 
organizational  authority  to  arbitrate  and  resolve 
disputes  that  may  arise  during  exchange.26 

This  view  sees  the  network  form  of  organization  as  an 
intermediate  between  hierarchies  and  markets,  where 
spontaneously  emerging  informal  links  supersede  the  formal 
organizational  chart.27  In  doing  so,  the  network 
organization  is  better  able  to  respond  to  the  "contingencies 
created  by  changing  markets,  technology,  and  other  facets  of 
a  chaotic  business  environment."28 

In  contrast,  other  scholars  such  as  Tichy,  Tushman,  and 
Fombrun  (1979),  and  Krackhardt  and  Carley  (1992),  view 
networks  as  something  that  resides  within  and  between 
organizations.  They  also  argue  that  network  analysis  tools 
should  be  used  to  explore  organizations  and  their  emergent 
structures  and  behavior  patterns.  In  fact,  some  scholars 
even  consider  the  network  itself  to  be  the  true  structure  in 
an  organization.29  This  perspective  acknowledges  that 
variations  in  the  prescribed  organizational  form  may  alter 

26  Joel  M.  Podolny  and  Karen  Page,  "Network  Forms  of  Organization, " 
Annual  Review  of  Sociology  (1998),  59. 

27  Stephan  P.  Borgatti  and  Pacey  C.  Foster,  "The  Network  Paradigm  in 
Organizational  Research:  A  Review  and  Typology,"  Journal  of  Management 
(2003),  995. 

28  Gregory  Bovasso,  "A  Structural  Analysis  of  the  Formation  of  a 
Network  Organization,"  Group  Organization  Management  (1992),  87. 

29  Ronald  S.  Burt,  Towards  a  Structural  Theory  of  Action:  Network 
Models  of  Social  Structure,  Perception,  and  Action  (New  York:  Academic 
Press ,  1982) . 
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the  characteristics  of  the  emergent  network,  but  insists 
that  other  factors  besides  the  formal  structure  critically 
impact  the  evolution  of  the  network.30 

E.  STATEMENT  OF  HYPOTHESES 

Pearce  and  David  (1983)  assert  that  social  network 
measures  can  be  used  to  test  the  performance  of  emergent 
structures  that  describe  alternative  interaction  patterns 
among  individuals  in  a  group.31  The  following  hypotheses 
test  the  performance  of  the  Hierarchy  in  complex  information 
processing  environment,  seeking  to  uncover  the  attributes  of 
performance  related  to  individuals  and  groups.  The 
information  processing  environment  created  by  the  ELICIT 
experiment  is  not  the  best  match  for  the  Hierarchy.  As  a 
result  a  mismatch  occurs  identical  to  mismatch  B,  identified 
by  Tushman  and  Nadler  (1978)  in  Figure  1,  where  the 
Hierarchy' s  information  processing  capacity  is  low,  while 
the  information  processing  requirements  are  extensive.  Yet, 
during  the  ELICIT  experiment,  the  Hierarchy  and  its  actors 
still  meet  with  success,  albeit  to  a  lesser  degree  than  do 
the  Edge  and  its  actors.  It  appears  that,  just  as  Pearce  and 
David  (1983)  assert,  the  macro  level  design  features  of 
ELICIT' s  Hierarchy  configuration  are  being  moderated  by 
group  level  characteristics.  Again,  the  subsequent 
hypotheses  test  for  the  relevance  of  these  characteristics 
within  ELICIT' s  Hierarchy  configuration. 


30  Fombrun  and  Tichy,  Human  Relations,  929. 

31  Pearce  and  David,  Academy  of  Management  Review,  437. 
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1. 


Individual-Level  Predictions 


The  positions  of  high  performing  individuals  are 
different  from  the  positions  of  low  performing  individuals 
within  a  group's  information  processing  structure . 

Every  session  of  the  ELICIT  experiment  results  in  the 
formation  of  a  web  of  communication  between  actors.  Each 
unique  network  represents  only  one  outcome  out  of  many 
possible  outcomes.  Each  actor  within  the  network  possesses  a 
set  of  nodal  characteristics  that  describe  his  relationships 
with  other  actors  and  the  network  itself.  The  figure  below 
is  a  representation  of  the  network  generated  by  ELICIT 
during  one  experimental  session.  Visually,  it  is  immediately 
apparent  that  individual  actors— represented  by  points— have 
differing  characteristics. 


powered  by  ORA,  CAS  OS  Center  @  CMU 

Figure  1.  Meta  Matrix  from  one  experimental  session. 
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a. 


Hi . 1 :  Degree  Centrality 


High  performing  individuals  are  more  likely  to  be 
found  in  the  center  of  an  organization's  communication 
network . 

Centrality  is  a  measure  of  the  extent  to  which  an 
actor  is  central  to  a  network.32  Wasserman  and  Faust  (1994) 
assert  that  an  actor  with  a  large  degree  centrality  is 
recognized  as  a  major  channel  of  relational  information  and 
a  crucial  cog  in  the  network.33  There  are  a  variety  of 
different  centrality  measures;  the  one  chosen  for  this 
hypothesis  is  degree  centrality.  ORA  defines  total  degree 
centrality  as  the  normalized  in-plus  out-degree  of  an  actor. 

Jb.  Hi .  2 :  Betweenness  Centrality 

A  high  degree  of  an  individual's  betweenness 
correlates  positively  with  high  performance . 

Betweenness  is  the  "extent  to  which  an  actor 
mediates,  or  falls  between  any  other  two  actors  on  the 
shortest  path  between  those  actors."34  An  actor  with  a  high 
degree  of  betweenness  is  considered  to  derive  power  by 
"controlling  or  brokering  the  flow  of  information."35 
Friedkin  (1991)  states  that  betweenness  indicates  increased 
interpersonal  influence.36 

32  Peter  R.  Monge  and  Noshir  S.  Contractor,  Theories  of  Communication 
Networks  (Oxford:  Oxford  University  Press,  2003),  32. 

33  Stanley  Wasserman  and  Katherine  Faust,  Social  Network  Analysis: 
Methods  and  Applications  (New  York:  Cambridge  University  Press,  1994), 
179. 

34  Monge  and  Contractor,  Theories  of  Communication  Networks,  32. 

35  Monge  and  Contractor,  Theories  of  Communication  Networks,  38. 

36  N.E.  Friedkin,  "Theoretical  Foundations  for  Centrality  Measures," 
American  Journal  of  Sociology  (1991),  1478-1504. 
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c. 


Hi . 3 :  Closeness  Centrality 


A  high  degree  of  an  individual's  closeness 
correlates  positively  with  high  performance 

Closeness  is  the  "extent  to  which  an  actor  is 
close  to,  or  can  easily  reach  all  the  other  actors  in  the 
network."37  An  actor  may  be  close  to  other  actors  by 
possessing  many  links  to  other  actors,  or  by  possessing 
links  to  actors  who  themselves  have  many  links  to  other 
actors.  Monge  and  Contractor  state  that  an  actor's  closeness 
measures  his  ability  to  receive  information  directly  or 
indirectly  "through  the  grapevine."38  Beauchamp  (1965) 
relates  that  actors  who  are  central  with  regard  to  closeness 
can  be  very  productive  in  communicating  information  to  other 
actors  . 39 

2 .  Team-Level  Predictions 

High  performing  groups  exhibit  characteristic  communication 
processing  patterns  different  from  low  performing  groups. 

Where  as  the  First  Substantive  Hypothesis  deals  with 
individual  actors  and  their  corresponding  attributes,  this 
hypothesis  examines  group  level  characteristic,  density. 

a.  H2 . 1 :  Group  Densi ty  vs .  Speed 

Higher  density  networks  submit  answers  in  less 
time  than  lower  density  networks . 


37  Monge  and  Contractor,  Theories  of  Communication  Networks,  32. 

38  Monge  and  Contractor,  Theories  of  Communication  Networks,  39. 

39  M.A.  Beauchamp,  "An  Improved  Index  of  Centrality, "  Behavioral 
Science  (1965),  161-163. 
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The  density  of  a  network  increases  as  number  of 
connections  between  actors  increases.  That  is  to  say,  the 
density  of  the  network  is  the  ratio  of  total  links  to 
possible  links  in  a  network.40  The  assumption  behind  this 
hypothesis  is  that,  in  a  denser  network,  more  relevant 
information  comes  to  actors  in  less  time,  which  results  in 
an  overall  decrease  in  the  time  necessary  to  make  decisions. 
Previous  studies  of  density  find  it  to  be  a  measure  of  group 
cohesion41  and  "knittedness."42 

Jb.  H2 . 2 :  Group  Density  vs.  Accuracy 

Higher  density  networks  submit  more  accurate 
answers  than  lower  density  networks . 

Because  higher  density  networks  possess  more 
avenues  through  which  information  is  passed,  this  hypothesis 
is  based  on  the  assumption  that  on  average  higher  density 
groups  are  empowered  to  make  more  accurate  decisions. 


40  Monge  and  Contractor,  Theories  of  Communication  Networks,  44. 

41  P.M.  Blau,  Inequality  and  Heterogeneity,  (New  York:  Free  Press, 
1977)  . 

42  J.A.  Barnes,  "Graph  Theory  and  Social  Network:  A  Technical  Comment 
on  connectedness  and  connectivity,"  Sociology  (1969),  215-232. 
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Ill .  METHODOLOGY 


The  following  description  of  method  outlines  the  means 
used  to  collect  the  data  for  this  thesis  and  the  method  by 
which  it  is  analyzed.  A  brief  description  of  the  ELICIT 
experiment  and  its  measures  enables  the  reader  to  better 
understand  the  nature  of  the  data  and  the  validity  of  the 
results.  Additionally,  the  reader  is  made  aware  of  the 
techniques  used  to  transmute  the  raw  data  generated  during 
the  ELICIT  experiment  into  the  form  required  for  analysis 
with  social  network  analysis  software. 

A.  INTRODUCTION  TO  ELICIT 

Originally,  the  ELICIT  experiment  was  conceived  to  test 
and  compare  the  performance  of  two  organizational  forms,  the 
Edge  and  the  Hierarchy  as  described  above  in  the  Literature 
Review.  Preliminary  results  are  available  in  Leweling  & 
Nissen  2007. 43  The  following  description  of  the  ELICIT 
experiment,  its  environment  and  measures,  are  based 
exclusively  on  this  paper. 

1.  ELICIT  Environment,  Organization,  and  Purpose 

Individuals  participating  in  the  ELICIT  experiment  are 
asked  to  identify  the  details  of  a  fictitious  terrorist 
plot,  specifically:  who,  what,  where,  and  when.  In  the 


43  Tara  A.  Leweling  and  Mark  E.  Nissen  offer  the  preliminary  results 
and  conclusions  of  this  investigation  in  a  paper  entitled  "Hypothesis 
Testing  of  Edge  Organizations:  Laboratory  Experimentation  using  the 
ELICIT  Multiplayer  Intelligence  Game"  presented  at  the  12th 
International  Command  and  Control  Research  and  Technology  Symposium: 
Adapting  C2  to  the  21st  Century  held  in  Newport,  Rhode  Island  in  June, 
2007  . 
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intelligence  game,  68  informational  clues  called  "factoids" 
are  distributed  throughout  the  group  of  17  players  over  a 
period  of  ten  minutes.  Each  player  receives  four  factoids, 
two  initially,  another  at  five  minutes,  and  the  final 
factoid  after  ten  minutes  of  game  play.  The  set  of  factoids 
that  each  player  receives  may  contain  information  relevant 
to  the  terrorist  plot,  but  each  set  is  arranged  so  that  no 
player  receives  enough  information  to  correctly  identity  the 
entire  solution.  Thus,  collaboration  becomes  necessary  among 
the  group  of  players  in  order  to  solve  the  problem 
correctly.  Presently,  there  are  four  versions  of  the  game, 
which  though  structurally  similar  are  each  composed  of  a 
unique  set  of  68  factoids.  More  versions  of  the  game  can  be 
created,  but  the  process  is  time  consuming  and  tedious. 

The  client  application  is  loaded  onto  separate, 

networked  computers  providing  each  player  with  a  set  of  five 

functions  needed  to  play  the  game:  1)  List,  2)  Post,  3) 

Pull,  4)  Share,  and  5)  Identify.  The  List  function  shows  all 

of  the  factoids  the  player  has  received,  either  from  the 

initial  distribution,  or  from  other  players  via  the  Share 

function.  Post  provides  players  with  a  common  screen  that 

displays  a  list  of  factoids  visible  to  multiple  players.  A 

player  takes  factoids  from  his  list  and  places  them  on  one 

of  the  four  different  Post  screens.  Contrarily,  a  player 

uses  Pull  to  take  factoids  from  one  of  the  common  screens 

and  add  them  to  his  list.  As  mentioned  above,  the  solution 

to  the  terrorist  plot  contains  four  parts— who,  what,  where, 

and  when— and  so,  the  four  Post  screens  contain  information 

pertinent  to  each  part  of  that  solution.  The  game  limits  the 

sharing  of  information  to  two  functions,  of  which  Post  is 

one  and  Share  is  the  other;  during  the  experiment,  no  verbal 
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communication  is  permitted.  In  order  to  pass  factoids  to 
specific  individuals,  a  player  first  selects  the  factoid  to 
be  sent  and  then  chooses  the  desired  recipient  from  a  list 
of  the  17  in-game  pseudonyms.  When  a  player  decides  that  he 
possess  the  correct  description  of  the  terrorist  plot,  he 
uses  the  Identify  function  to  enumerate  the  details,  i.e., 
who,  what,  where,  and  when. 

Throughout  the  game,  the  server  application  records  and 
time  stamps  transaction  data  on  text-file  logs.  Nearly  every 
activity  that  takes  place  in  the  game  is  registered  on  these 
log  files,  including  when,  which,  and  to  whom  factoids  are 
distributed,  posted,  pulled,  viewed,  and  shared.  The  log 
files  also  record  when  and  what  each  player  identifies  as 
his  solution.  Researchers  then  process  these  files  in  any 
number  of  ways  in  order  to  distill  the  information  which 
they  desire.44  For  this  thesis  the  author  coded  data  from 
the  log  files  into  time  demarcated  adjacency  matrices  which 
were  then  evaluated  with  the  Organizational  Risk  Analyzer 
(ORA) ,  a  social  network  analysis  software  application 
developed  at  Carnegie  Mellon  University.45 

2 .  Operationalizing  the  Edge  and  Hierarchy 

To  test  and  compare  the  performance  of  the  Hierarchy 
and  the  Edge  experimental  environments  are  created  with  the 
ELICIT  software  that  are  modeled  after  both  organizational 
forms.  Firstly,  regardless  of  the  configuration,  a  subject 
is  assigned  to  one  of  four  groups.  Again,  the  groups 

44  Leweling  and  Nissen,  12th  ICCRTS,  5. 
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correspond  with  a  part  of  the  Identify  function;  there  is  a 
"who"  group,  a  "what"  group,  and  so  on.  While  playing  in  the 
Edge  configuration,  subjects  are  allowed  to  interact  with 
all  of  the  post-pull  common  screens.  Specifically,  a  member 
of  the  "who"  group,  in  addition  to  his  own  screen,  can  Pull 
from  and  Post  to  the  "what"  screen,  the  "where"  screen,  and 
the  "when"  screen.  The  situation  while  in  the  Hierarchy 
configuration  is  dramatically  different.  To  begin  with, 
there  is  an  overall  group  leader  (i.e.  labeled  "1")  who  is 
responsible  for  the  intelligence  organization  as  whole. 
Reporting  directly  to  him  are  four  functional  leaders  (i.e. 
labeled  "2,"  "6,"  "10,"  and  "14")  each  of  whom  is  in  charge 
of  the  three  analysts  assigned  to  his  group.  The 
interactions  between  these  groups  is  limited  in  as  much  as  a 
member  of  the  "who"  group,  for  instance,  is  only  able  to 
Pull  from  and  Post  to  the  "who"  common  screen.  Likewise,  but 
for  the  over  all  group  leader  who  has  access  to  all  four 
screens,  every  other  player  has  truncated  access  to  the 
common  screens.  However,  players  are  still  able  to  pass 
factoids  to  each  other  via  the  Share  function.46 


45  Kathleen  M.  Carley,  ORA:  the  Organizational  Risk  Analyzer 
(Pittsburgh,  Pennsylvania:  Carnegie  Mellon  University,  School  of 
Computer  Science,  Institute  for  Software  Research  International,  Center 
for  Computational  Analysis  of  Social  and  Organizational  Systems,  2007) . 

46  Leweling  and  Nissen,  12th  ICCRTS,  8. 
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Who  Group  What  Group  Where  Group  When  Group 

Figure  2.  Hierarchy  Organization  from  Leweling  and  Nissen 

The  ELICIT  experiments  employ  an  additional  means  of 
communication  for  players  which  for  the  purposes  of  this 
thesis  has  been  treated  as  another  type  of  sharing.  Players 
share  Postcards  with  each  other  several  times  throughout  the 
game.  Players  pick  to  whom  to  send  their  Postcards,  but 
their  options  are  limited  while  in  the  Hierarchy 
configuration;  a  subject  is  only  permitted  to  share 
postcards  with  members  of  his  own  group,  while  functional 
leaders  are  allowed  to  share  postcards  with  the  overall 
leader,  and  the  overall  leader  can  share  them  with  anyone. 
While  factoids  represent  the  sharing  of  information. 
Postcards  provide  a  snapshot  of  how  a  player  understands  the 
information  at  a  certain  moment  in  time.  That  is  to  say,  to 
share  a  Postcard  is  to  convey  one's  knowledge.47  Further 
information  about  the  use  of  Postcards  in  the  ELICIT 
experiments  and  the  results  of  the  hypotheses  associated 
with  them  can  be  found  in  aforementioned  paper  by  Leweling 
and  Nissen. 


47  Leweling  and  Nissen,  12th  ICCRTS,  10-12. 
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B. 


FROM  LOG  FILES  TO  INDEPENDENT  VARIABLES 


In  all,  eight  of  the  sixteen  log  files  generated  during 
the  ELICIT  experiment  are  analyzed  in  this  thesis, 
specifically  the  ones  having  to  due  with  the  hierarchy 
configuration.  As  mentioned  above,  each  log  file  notes  the 
time  and  type  of  interaction  that  occurs  between  subjects. 
These  events  are  displayed  in  a  text  document  where  on  each 
line  is  recorded  the  specifics  of  the  event:  who,  what,  and 
when.  Experimental  sessions  last  approximately  one  hour;  by 
the  end  of  a  session  the  log  file  generated  is  approximately 
one  hundred  and  fifty  pages  in  length  (using  the  formatting 
of  this  thesis)  .  Before  extracting  the  data  from  the  log 
files  into  adjacency  matrices,  they  are  converted  from  text 
files  into  Excel  notebooks  which  allow  for  sorting  of  agent 
names,  event  types,  and  times.  This  enables  a  much  more 
efficient  transfer  of  the  data. 

In  order  to  analyze  the  data  in  the  long  files  with 
social  network  analysis  software,  one  must  create  an 
adjacency  matrix  for  each  file.  An  adjacency  matrix 
possesses  an  equal  number  of  rows  and  columns,  as  both  are 
headed  by  the  same  the  names  of  the  agents  to  be  studied.  In 
the  example  below  notice  that  the  names  in  the  column 
headings  represent  the  receivers  of  the  information  sent  by 
the  names  in  the  first  column: 


Alex  Chris 

Dale 

Francis  Harlan  Jesse  Kim 

Leslie  Morgan  Pat 

Robin 

Alex 

Chris 

Dale 

Francis 

1 

Harlan 

Jesse 

2 

1 

2  1 

Kim 

1 1 

Leslie 

4 

3  2  3 

|  1 

Morgan 

1 

2 

1 

2 

1  1 

Pat 

3 

Robin 

1 

Table  2.  Example  of  Adjacency  Matrix 
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The  adjacency  matrices  are  first  compiled  on  and  saved 
in  regular  Excel  notebooks.  Every  three  hundred  second  time 
interval  receives  its  own  matrix  within  the  session' s  Excel 
notebook.  Eight  log  files  result  in  eight  notebooks,  but 
each  notebook  contains  a  maximum  of  thirteen  adjacency 
matrices.  In  addition  to  facilitating  more  rapid  error 
checking,  this  break  down  allows  researchers  to  study 
network  development  over  time.  After  the  Excel  notebooks  are 
completed  the  matrices  are  individually  copied  and  pasted 
into  new  spreadsheets  and  saved  in  the  comma  delimited 
format  required  by  ORA.  Next,  the  matrices  are  loaded  into 
ORA  so  that  the  matrices  from  one  experimental  session  are 
saved  together  in  one  meta  matrix. 

The  process  of  converting  these  matrices  into 
independent  variables  begins  with  use  of  matrix  algebra.  ORA 
makes  possible  the  addition  of  different  matrices,  in  this 
case  the  summation  of  the  thirteen  adjacency  matrices 
constructed  for  each  experimental  session.  The  resulting 
summation  is  a  complete  catalog  of  every  interaction  that 
occurred  during  one  experimental  session  and  the  record  of 
data  from  which  the  independent  variables  are  derived. 

Finally,  the  researcher  selects  the  desired  social 
network  measures  from  the  long  list  provided  by  ORA  and  the 
software  calculates  the  desired  measures  which  are  the 
independent  variables.  Again,  this  thesis  examines  the 
individual  level  measures,  betweenness,  centrality,  and 
closeness,  and  the  group  level  measure  density.  Chapter  IV 
details  the  specific  formulas  used  to  calculate  the 
independent  variables  for  each  hypothesis. 
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C.  DEFINITION  OF  HIGH  PERFORMANCE 

Each  hypothesis  evaluated  in  this  thesis  is 
statistically  compared  to  a  measure  of  success  or 
performance  with  the  network  measures  being  the  independent 
variables,  while  the  measures  of  performance  being  the 
dependant  variables.  It  is  therefore  essential  to  clearly 
define  high  performance  and  describe  its  constituent 
dimensions  within  the  ELICIT  environment.  In  order  to 
transpose  the  results  of  this  thesis  and  make  claims  upon 
the  real  world,  the  subsequent  descriptions  of  high 
performance  must  be  based  on  legitimate  measures. 

1 .  Speed 

The  first  component  of  success  is  the  time  it  takes  for 
a  player  to  submit  his  identification  of  the  terrorist  plot. 
Group  performance  then  becomes  the  mean  submission  times  of 
all  players  participating  in  the  experimental  session.  For 
ease  of  comparison,  time  scores  are  normalized  to  a  0-1 
scale,  with  1  representing  the  fastest  time  to  submission. 
Because  all  identifications  are  time  stamped  in  the  log 
files,  measurements  are  easy  to  construct.  In  order  to 
compare  times  meaningfully  from  one  session  to  another,  the 
clock  times  of  all  sessions  are  considered  to  be  equivalent. 
In  other  words,  a  submission  at  2200  seconds  after  the  start 
of  Session  1  is  considered  to  be  exactly  as  fast  as  a 
submission  at  2200  seconds  after  the  start  of  Session  2. 
Each  subjects  normalized  identification  time  is  thus  derived 
from  Equation  1,  where  3896  represents  the  maximum  time 
elapsed  during  experimental  sessions: 
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3896  -  identification  time 

Equation  1:  T  = - - - = - 

3896 


For  example,  in  order  to  calculate  the  normalized 
identification  time  for  a  submission  occurring  at  2200 
seconds  after  the  start  of  any  experimental  session,  one 
substitutes  as  has  been  done  below:48 

3896-2200  ^ 

- =  .44 

3896 


2 .  Accuracy 

The  second  component  of  performance,  accuracy,  is 
measured  by  awarding  one  point  to  a  correct  submission  for 
each  part  of  the  identification— who,  what,  where,  and  when— 
and  normalizing  the  score  on  a  0-1  scale  giving  equal  weight 
to  each  part.  Thus,  a  perfectly  accurate  submission,  where 
all  four  parts  are  correct,  receives  a  1,  while  a  completely 
inaccurate  submission  receives  a  0.  Again,  the  group 
performance  is  the  mean  accuracy  of  every  participant's 
identification.49 

D.  STATISICAL  ANALYSIS 

Once  the  independent  and  dependent  variables  are 
calculated,  the  researcher  must  determine  which  statistical 
tests  are  appropriate  for  comparing  the  variables  and 
determining  the  existence  of  correlations. 


48  Leweling  and  Nissen,  12th  ICCRTS,  10-11. 

49  Leweling  and  Nissen,  12th  ICCRTS,  11. 
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1. 


Individual -Level  Measures 


Each  Individual-Level  hypothesis  tests  for  a 
correlation  between  a  nodal  characteristic  (the  independent 
variable)  and  individual  performance  (the  dependent 
variable) .  Using  the  Kolmogorov-Smirnov  test  for  normality, 
it  is  determined  with  a  high  degree  of  certainty  that  that 
the  independent  variables  are  not  normally  distributed 
(i.e.,  p<  0.05  for  time,  and  p  <  0.05  for  accuracy)  .  It  is 
therefore  appropriate  when  testing  these  hypotheses  to  use 
non-parametric  tests  for  correlations,  in  this  case  a 
Kendall's  Tau  B.  Because  the  hypotheses  are  directional, 
i.e.  testing  whether  higher  centrality  correlates  to  higher 
performance,  the  tests  are  also  one  sided. 

2 .  Team-Level  Measures 

The  two  Team-Level  hypotheses  compare  the  performance 
of  teams.  While  the  data  for  the  Individual-Level  Hypotheses 
are  not  normally  distributed,  with  the  Kolmogorov-Smirnov 
test  for  normality  it  is  determined  that  this  data  is 
normally  distributed  (i.e.  p>.05)  and  that  it  is  therefore 
appropriate  when  testing  these  hypotheses  to  use  Pearson''  s 
tests  for  correlations. 

E .  SUMMARY 

This  chapter  describes  the  ELCIT  environment,  largely 

borrowing  from  Leweling  and  Nissen  (2007),  and  proceeds  to 

outline  the  techniques  used  to  convert  the  data  recorded 

during  the  experiment  into  reliable  measures  of  centrality 

and  density,  the  independent  variables  analyzed  in  this 

thesis.  Performance,  in  terms  of  time  and  accuracy,  are  the 
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dependent  variables  of  this  thesis.  Finally,  the  independent 
and  dependent  variables  are  checked  for  correlations.  The 
results  of  these  efforts  are  described  in  the  subsequent 
chapter . 
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IV.  RESULTS  AND  ANALYSIS 


This  chapter  presents  the  results  and  analysis  of  the 
hypotheses  introduced  in  Chapter  II.  It  is  important  for  the 
reader  to  remember  that  this  thesis  examines  two  different 
entities:  individuals  and  teams.  Section  A  of  this  chapter 
presents  the  findings  of  Individual-Level  analysis,  while 
Section  B  covers  Team-Level  analysis.  A  discussion  of  the 
results  occurs  in  Section  C. 

A.  INDIVIDUAL-LEVEL  OF  ANALYSIS 

Because  performance  within  the  ELICIT  environment 
contains  two  components— time  and  accuracy— correlations  exist 
sometimes  between  a  nodal  characteristic  and  both  time  and 
accuracy,  sometimes  exclusively  time  or  accuracy,  and 
sometimes  neither  time  nor  accuracy. 

1 .  HI . 1 :  Degree  Centrality 

The  first  hypothesis  is  accepted  in  part,  as  a 
correlation  exists  between  an  actor' s  degree  centrality  and 
his  answer's  accuracy.  Degree  centrality  is  calculated  using 
the  equation  below,  where  i  and  j  are  both  actors  in  a 
matrix,  X,  with  n  entities: 

^  n  n 

Equation  2:  mtity  ’  =  2(n^V)  ^  £  V  ‘ ‘  ' 

/ i^\ 

Degree  centrality  and  accuracy  are  correlated  (r  =  .191,  p  < 
0.01) .  Using  a  linear  model,  degree  centrality  accounts  for 

50  Wasserman  and  Faust,  Social  Network  Analysis,  199. 
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3.6%  of  the  variance  in  accuracy.  Degree  centrality  is  thus 
not  a  useful  predictor  of  accuracy.  On  the  other  hand, 
degree  centrality  and  time  to  submission  do  not  appear  to 
correlate  (r=.053,  p>  0.01).  Therefore,  it  appears  that  a 
relationship  exists  between  being  positioned  in  the  center 
of  a  network  and  a  more  accurate  answer,  but  it  cannot  be 
said  that  the  relationship  is  predictive. 

2.  HI . 2 :  Betweenness  Centrality 

A  correlation  exists  between  the  independent  variable, 
betweenness,  and  an  individual's  performance,  both  accuracy 
and  time  to  submission.  The  betweenness  centrality  of  entity 
v  in  a  network  is  defined  as:  across  all  entity  pairs  that 
have  a  shortest  path  containing  v,  the  percentage  that  pass 
through  v.51  The  correlation  coefficient  for  betweenness  and 
accuracy  is  .164  with  pc.01,  while  the  correlation 
coefficient  for  betweenness  and  time  to  submission  is  .113 
with  p<.05.  It  appears  that  there  is  very  little  common 
variation  between  the  independent  variable  and  performance- 
both  accuracy  and  time  (4%  and  1.2%,  respectively),  but  that 
a  statistically  significant  relationship  exists. 

3.  HI. 3:  Closeness  Centrality 

Like  Hl.l,  an  actor's  closeness  correlates  with  the 
accuracy  component  of  performance,  but  not  the  time  to 
submission  component.  An  actor's  closeness  is  calculated 
with  the  following  equations,  where  G=(V,E)  is  the  graph 
representation  of  the  square  network: 


51  Carley,  "Measures,"  ORA  Help  Manual,  Centrality,  Betweenness. 
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Equation  3: 


V  - 


(M-i) 


di  st 


Equation  4 


dist  -  I  dG(v,i) 
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For  accuracy,  r=.154  and  p<.05  and  for  time  to 
submission,  r=.013  and  p>.05.  Aqain,  a  relationship  exists 
between  the  independent  variable,  closeness,  and  the 
dependant  variable,  accuracy,  but  that  relationship  is  not 
predictive . 

B.  TEAM- LEVEL  ANALYSIS 

The  two  subsequent  hypotheses  test  for  a  correlation 
between  the  independent  variable,  density,  and  the  dependant 
variables,  speed  and  accuracy. 

1 .  H2 . 1 :  Group  Density  vs .  Speed 

The  most  surprisinq  result  of  this  analysis  is  the 
correlation  between  network  density  and  performance.  Not 
only  must  the  null  hypothesis  be  rejected  as  untenable,  but 
in  fact  it  is  determined  that  a  correlation  does  exist  which 
is  exactly  opposite  of  what  was  assumed;  namely  that  as 
network  density  increases  time  to  submission  increases  as 
well.  One  possible  explanation  for  this  outcome  is  that  as 
network  density  increases  actors  require  more  time  to 
process  information  and  manaqe  the  larqer  number  of 
connections  to  other  actors.  The  Pearson's  correlation 
between  network  density  and  time  to  submission  is  -.834  with 
p= . 005 . 

52  Carley,  "Measures,"  ORA  Help  Manual,  Centrality,  Closeness. 
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2. 


H2 . 2 :  Group  Density  vs .  Accuracy 


From  the  result  of  H2.1,  it  can  be  inferred  that 
because  actors  functioning  in  higher  density  networks  are 
forced  to  process  an  increased  amount  of  information,  it 
takes  them  more  time  to  submit  answers.  One  might  suspect 
that  the  additional  time  and  information  associated  with 
higher  network  densities  would  result  in  more  accurate 
submissions,  but  like  H2.1,  H2 . 2  must  be  rejected.  There  is 
no  statistically  significant  correlation  between  network 
density  and  the  accuracy  of  answers.  The  Pearson's 
correlation  between  network  density  and  accuracy  is  .022 
with  p=.479,  clearly  an  ambiguous  result,  and  therefore 
nothing  can  be  said  with  certainty.  At  most,  when  looking  at 
the  scatter  plot,  one  notices  a  slight  upward  trend  in 
accuracy  as  density  increases;  at  least  in  this  case  it  is 
in  the  direction  of  what  had  been  supposed. 

C.  DISCUSSION 

The  three  hypotheses  dealing  with  Individual-Level 
measures  have  in  common  the  idea  of  network  centrality;  that 
an  actor  who  is  central  to  a  network  should  experience  an 
advantage  in  terms  of  performance  over  actors  who  are  less 
central.  Although  in  all  cases  a  statistically  significant 
relationship  exists  between  centrality  and  accuracy,  only 
betweenness  centrality  exhibits  a  relationship  with  speed. 
Unfortunately,  none  of  the  network  measures  examined  in  this 
thesis  are  useful  predictors  of  an  individual's  performance. 
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explanation  for  the  unexpected  result  of  H2 . 1  is  correct- 
that  as  network  density  increases,  on  average,  actors 
require  more  time  to  submit  answers— then  it  indeed  seems 
plausible  that  the  demand  which  managing  more  connections 
places  on  actors  who  are  more  central  to  a  network  lessens 
some  of  the  advantages  gained  from  that  location.  That  is  to 
say,  more  central  actors,  may  provide  more  accurate 
solutions  on  account  of  their  position,  but  that  the  added 
demands  placed  on  the  position  detract  from  some  of  its 
benefits . 

Based  on  these  assumptions,  it  seems  that  the  highest 
performing  individuals  should  possess  a  careful  balance 
between  a  central  position  and  a  larger  number  of  links. 
Although  the  former  may  provide  better  access  to 
information,  the  latter  may  tend  to  overwhelm.  Perhaps 
centrality  exhibits  a  diminishing  return;  each  additional 
link  which  makes  an  actor  more  central  to  a  network  incurs  a 
higher  cost— in  terms  of  time  and  effort— than  previously 
established  links.  For  these  conjectures  to  be  true  two 
assumptions  must  be  verified.  Firstly,  centrality  must 
indeed  be  a  boon  to  an  actor' s  information  processing 
capacity— an  assertion  made  by  social  network  analysis 
literature,  but  only  weakly  supported  by  the  results  of  this 
thesis.  And  secondly,  an  excessive  number  of  links  must 
indeed  hinder  an  actor's  performance— which,  in  this  case,  is 
an  intuitive  claim  concerning  human  capacity  supported  by 
psychology . 

As  an  explanation,  this  thesis  proposes  that  as  network 
density  increases  actors  require  more  time  to  process  and 
respond  to  incoming  information.  In  as  much  as  central 
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greater  number 


of  edges  ( i . e . , 


actors  possess  a 
communication  linkages  to  others) ,  this  thesis  also  argues 
that  centrality  in  a  network  has  costs,  as  well  as  benefits. 
Further  experimentation  is  needed  to  test  the  validity  of 
these  conjectures  and  bring  better  understanding  to 
Organization  Theory,  Social  Network  Analysis,  and 

Information  Processing  networks. 

In  sum,  the  results  of  this  thesis  present  findings 
that,  though  inconclusive,  call  into  question  intuition  and 
social  network  analysis  literature.  Perhaps,  improved 
techniques  and  more  subtle  measures  are  necessary  in  order 
to  reconcile  predictions  with  observations.  It  is  clear, 
however,  that  the  dynamic  web  of  interaction  created  by  the 
ELICIT  experiment  involves  more  complexity  than  originally 
had  been  anticipated;  and  this  added  complexity  offers 
future  researchers  a  challenge  and  the  opportunity  for 
greater  understanding. 
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V.  IMPLICATIONS  FOR  THEORIES  AND  RESEARCH 


This  chapter  begins  with  discussion  of  the  study' s 
results  for  Command  and  Control,  Organizational  Design, 
Information  Processing,  and  Social  Network  Analysis.  A  brief 
description  of  the  Edge  organization  follows  and  is  provided 
for  future  researchers  who  may  be  interested  in  analyzing 
and  comparing  the  rest  of  the  data  collected  during  the 
ELICIT  experiment.  Next,  opportunities  for  future  research 
are  discussed.  Finally,  the  reader  is  presented  with  a  brief 
summery  of  the  thesis's  findings  and  conclusions. 

A.  IMPLICATIONS  FOR  MILITARY  COMMAND  AND  CONTROL 

Because  the  ELICIT  experiment  was  modeled  to  be  an 
intelligence  game,  the  results  of  this  thesis  most  readily 
operationalize  to  units  which  collect  and  analyze  military 
intelligence.  More  generally,  however,  the  results  of  this 
thesis  seem  to  reinforce  the  idea  of  presenting  human  beings 
with  a  limited  and  orderly  flow  of  information  when 
designing  command  and  control  systems.  Rather  than 
encouraging  individuals  to  seek  out  too  much  information, 
command  and  control  systems  engineers,  by  means  of  their 
design,  ought  to  prevent  the  user  from  becoming  burdened 
with  superfluous  information. 

B.  IMPLICATIONS  FOR  ORGANIZATIONAL  DESIGN 

It  is  important  to  remember  that  the  entities  studied 
in  this  thesis  are  individuals,  and  that  human  decision 
making  enjoys  characteristics  and  capacities  that  are  far 
different  from  those  of  automated  decision  making.  While 
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human  beings  seem  to  have  difficulty  managing  an  abundance 
of  links,  computers  are  ever  increasingly  capable.  When  it 
comes  to  designing  the  macro-level  of  features  of  an 
organization,  the  results  of  this  thesis  indicate  that  human 
capacity  must  be  considered  when  attempting  to  maximize 
performance;  human  exposure  to  sources  of  information  should 
be  limited  in  some  fashion.  Theorists  must  be  careful  not  to 
overemphasize  the  hierarchies'  stove-pipes  while  neglecting 
its  ability  parcel  information  into  quantities  human  beings 
find  manageable. 

C.  IMPLICATIONS  FOR  INFORMATION  PROCESSING  THEORY 

In  as  much  as  the  organizations  formed  during  the 
ELICIT  experiment  can  be  seen  as  information  processing 
units,  it  would  be  helpful  to  understand  the  difference 
between  information  and  knowledge  processing.  The  networks 
in  this  thesis  are  created  by  merging  the  information 
networks  (factoids)  and  knowledge  networks  (postcards) . 
Preliminary  results  from  Leweling  and  Nissen  (2007)  indicate 
that  in  some  situations  the  sharing  of  mental  models 
increases  the  time  to  submission.53  Perhaps  a  better 
understanding  and  more  meaningful  measure  would  result  from 
separating  and  analyzing  the  two  networks  with  similar 
methods  used  in  this  thesis.  The  effect  of  centrality  in  a 
knowledge  network  may  not  be  the  same  as  the  effect  of 
centrality  in  an  information  network. 


53  Leweling  and  Nissen,  12th  ICCRTS,  14. 
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D.  IMPLICATIONS  FOR  USING  NETWORK  ANALYSIS  TO  EXPLORE 

ORGANIZATIONS 

Although  widely  accepted  as  gauge  of  an  actor' s 
importance  to  a  network,  the  results  of  this  thesis  only 
weakly  suggest  that  centrality  relates  to  higher 
performance.  A  great  deal  more  work  must  be  done  in  order  to 
truly  understand  the  subtleties  and  conditions  of  this 
property.  Centrality  seems  to  confer  benefits,  but  not 
without  costs.  The  results  of  the  Team-Level  hypotheses 
(H2.1)  suggest  that  there  exists  an  optimal  performance 
density  for  organizations  facing  conditions  analogous  to  the 
one  created  by  the  ELICIT  experiment.  From  these  results,  it 
can  be  assumed  that  a  maximally  dense  network  will  not 
exhibit  maximum  performance.  Therefore,  it  would  be 
advantageous  to  know  if  an  optimal  performance  density 
exists  and  into  what  range  falls. 

E.  OPPORTUNITIES  FOR  FUTURE  RESEARCH 

It  is  important  to  remember  that  the  ELICIT  environment 
provides  researchers  with  the  opportunity  to  experiment  in 
the  laboratory  with  group  sizes  that  have  been  generally 
reserved  for  field  studies.54  ELICIT  can  be  used  to  examine 
the  effects  of  a  host  of  different  forces  which  act  on 
organization,  including:  incentive  structure,  culture,  and 
the  role  of  planning  and  strategy.  This  thesis  is  limited  to 
a  study  of  the  attributes  of  successful  performance  in  a 
complex  and  interdependent  information  processing 


54  H.  Kang,  H.  Yang,  and  C.  Rowley,  "Factors  in  team  effectiveness: 
Cognitive  and  demographic  similarities  of  software  development  team 
members,"  Human  Relations  (December,  2006),  1681-1711. 
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environment,  and  possible  related  areas  of  study  abound.  For 
instance,  this  thesis  investigated  only  the  half  of  the  data 
collected  during  the  ELICIT  experiment  having  to  do  with  the 
Hierarchy  configuration.  The  other  half  of  the  data  having 
to  do  with  the  Edge  configuration  remains  unexamined  by 
graph-theoretic  technigues.  A  willing  researcher  could  test 
exactly  the  same  hypotheses  as  the  ones  developed  for  this 
thesis  and  compare  the  results.  In  light  of  the  findings 
having  concerning  density,  a  researcher  could  design  an 
experiment  to  test  for  the  optimal  density  levels  discussed 
in  Section  D  of  this  chapter.  The  experiment  could  control 
for  the  number  of  links  an  actor  is  allowed  to  establish 
during  game  play  by  dictating  maximums  and  minimums  to 
different  groups  of  players,  which  would  serve  to 
artificially  adjust  density  and  other  network  measures. 
Another  experiment  could  be  created  to  study  how  different 
organizational  forms  respond  to  dynamic  situations  by 
changing  mission  requirements  during  game  play.  To  add 
further  incite  into  the  role  of  latent  actor  traits  in 
network  development,  an  experiment  could  be  conducted  which 
compares  nodal  characteristics  with  personality  traits.55 
In  all,  ELICIT  is  a  valuable  tool  for  exploring  groups  and 
individuals,  and  should  provide  creative  researchers  with  a 
means  to  answer  complex  questions  about  organizations. 

F .  SUMMARY 

By  using  of  graph-theoretic  techniques  to  compare  the 
information  processing  behaviors  of  three  groups  of  mid¬ 
level  working  professionals  as  each  undertakes  a  series  of 


55  Borgatti  and  Foster,  Journal  of  Management  Review,  1000. 
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four  complex,  interdependent,  computer-mediated  decision¬ 
making  exercises,  this  thesis  explores  1)  the  location 
within  the  information  processing  structure  of  high- 
performing  individuals  v.  low-performing  individuals,  and  2) 
the  exhibited  characteristic  communication  processing 
patterns  of  high-performing  groups  v.  low-performing  groups. 
The  results  of  this  exploration,  though  mostly  inconclusive, 
call  into  question  both  intuition  and  social  network 
analysis  literature.  It  is  predicted  that  centrality  in  a 
network  correlates  positively  with  high  performance  among 
individuals;  but  statistics  reveal  an  almost  negligible 
relationship.  It  is  also  hypothesized  that  high  density 
groups  outperform  low  density  groups,  but  density  and 
performance  were  found  to  correlate  in  exactly  the  opposite 
direction:  as  density  increases,  performance  decreases. 

As  an  explanation,  this  thesis  proposes  that  as  network 
density  increases  actors  require  more  time  to  process 
information  and  manage  the  larger  number  of  connections  to 
other  actors.  In  as  much  as  central  actors  possess  more 
links  than  actors  on  the  periphery  of  a  network,  this  thesis 
also  argues  that  centrality  in  a  network  has  costs,  as  well 
as  benefits,  and  sometimes  the  demands  of  managing  too  many 
links  can  distract  an  individual  from  the  task  at  hand. 
Further  experimentation  is  needed  to  test  the  validity  of 
these  conjectures  and  bring  better  understanding  to  the 
fields  of  Organization  Theory  and  Social  Network  Analysis. 
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APPENDIX:  META  MATRICES 


powered  by  ORA,  CASOS  Center  @  CMU 


powered  by  ORA,  CASOS  Center  @  CMU 


powered  by  ORA,  CASOS  Center  @  CMU 
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